DOCUMENT RESUME 



TM 035 215 

Kim, Se-Kang; Davison, Mark L. 

Connecting SEM Analysis and Profile Analysis via MDS. 

2003-04-00 

21p.; Paper presented at the Annual Meeting of the American 
Educational Research Association (Seattle, WA, April 2001) . 

Reports - Research (143) — Speeches/Meeting Papers (150) 

EDRS Price MFOl/PCOl Plus Postage. 

*Adults; Factor Analysis; Factor Structure; Interest Inventories; 
^Multidimensional Scaling; Profiles; ^Structural Equation Models 
^Profile Analysis Approach 



This study was designed to explain how Profile Analysis via 
Multidimensional Scaling (PAMS) could be viewed as a structural equations model (SEM) . 
The study replicated the major profiles extracted from PAMS in the context of the 
latent variables in SEM. Data involved the Basic Theme Scales of the Strong Campbell 
Interest Inventory (Campbell and Hanse,. 1985). Data were collected from 1, 308 males, 
who were clients of a vocational assessment clinic. Findings show that the profile 
patterns identified by PAMS can be used to examine the relationships between the 
profile patterns and other criterion variables, but the results from a PAMS approach 
can only be used as an exploratory tool. The profile patterns can be examined in 
confirmatory factor analysis as shown by this study. Also, when major profiles 
identified in PAMS are replicated in SEM, one can examine whether significant test 
results of the coordinates in PAMS match those of factor loadings in SEM, since SEM 
analyses always provide asymptotic standard errors corresponding to estimates of 
parameters. (Contains 2 figures, 3 tables, and 14 references.) (SLD) 



ED 480 138 

AUTHOR 
TITLE 
PUB DATE 
NOTE 

PUB TYPE 
EDRS PRICE ' 
DESCRIPTORS 

IDENTIFIERS 

ABSTRACT 




Reproductions supplied by EDRS are the best that can be made 
from the original document. 



TM035215 



I 



Connecting SEM and PAMS 



Running Head: CONNECTING SEM AND PAMS 



oo 

m 

o 

oo 

Q 

w 



Connecting SEM Analysis and Profile Analysis via MDS 



Se-Kang Kim 

University of Missouri-Columbia 



Mark L. Davison 

University of Minnesota-Twin Cities 



PERMISSION TO REPRODUCE AND 

DISSEMINATE THIS MATERIAL HAS 

BEEN GRANTED BY 

Klirvt 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 

1 



^ U.S. DEPARTMENT OF EDUCATION 
Office of Educational Research and Improvement 
EDUCATIONAL RESOURCES INFORMATION 
CENTER (ERIC) 

his document has been reproduced as 
received from the person or organization 
originating it. 

□ Minor changes have been made to 
improve reproduction quality, 



® Points of view or opinions stated in this 
document do not necessarily represent 
official OERI position or policy. 



Paper prepared for the American Educational Research Association Conference, Seattle, WA, 
April 11, 2001. Correspondence concerning the manuscript should be addressed to Se-Kang 
Kim, Department of Educational and Counseling Psychology, University of Missouri, 16 Hill 
Hall, Columbia, MO 65211. 




BEST COPY AVAILABLE 

9 






1 



Connecting SEM and PAMS 



Connecting SEM Analysis and Profile Analysis via MDS 
Se-Kang Kim Mark L. Davison 

University of Missouri University of Minnesota 

This paper was designed to explain how the Profile Analysis via Multidimensional 
Scaling (PAMS) model could be viewed as a structural equations model (SEM). In the previous 
presentation (Davison, Kim, & Ding, 2001), the PAMS model was used in an exploratory MDS 
analysis for identifying major profiles. This paper presented a parallel analysis of the same data 
but attempted to develop a confirmatory approach by means of the structural equations model 
under the assumptions of the PAMS model. One may speculate that there is a connection 
between latent variables in SEM and dimensions in MDS, and easily conclude that the latent 
variables in SEM correspond MDS dimensions. MDS dimensions are actually major profiles in 
PAMS, but PAMS emphasizes interpretation of patterns in dimensions. In the paper, we 
replicated the major profiles extracted from PAMS in the context of the latent variables in SEM. 

A line of research (e.g.. Rounds, Davison, & Dawis, 1979; Tracy & Rounds, 1993) by 
James Rounds suggests that it is possible to use structural equations modeling to develop a 
confirmatory approach for a MDS analysis. Rounds has been studying the structure of 
vocational interests. Although Rounds has not exploited the profile pattern interpretation of his 
results, he began using multidimensional scaling to explore the core dimensional profile patterns 
that underlie vocational interests (Rounds, Davison, & Dawis, 1979). Using a relationship 
between factor analysis and multidimensional scaling developed by Davison (1985), Rounds has 
developed a confirmatory analysis of covariance structures (e.g., Tracy & Rounds, 1993). 
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The combination of the PAMS model and Rounds’ work implies that by considering the 
linear components of measurement around profile patterns, rather than latent factors, it is 
possible to connect the analysis of covariance structures to the study of profile patterns, if a 
researcher intents to study profiles defined over variables that have been standardized in the 
research sample. 

To specify the PAMS model with K major profile patterns as a structural equations 
model, one must specify a structural equations model with 1 +K latent variables. The first 
additional latent variable corresponds to the level parameter in the PAMS model and the last K 
latent variables correspond to the K major profile patterns in PAMS. As introduced in the 
previous presentation (Davison et a., 2001), the level parameter accounts for each individual’s 
profile height and determines how much the individual’s profile elevated or depressed from the 
major profiles. The level parameter is estimated by the individual’s total score over subscales. 

Then one may ask why SEM analysis includes one more additional latent variable than 
PAMS. To answer, it is necessary to examine the relation between parameter estimates in the 
factor model and the PAMS model. Davison (1985) compared MDS analysis with principal 
components analysis (PCA), using the same test intercorrelations. In his Monte Carlo studies, 
Davison replicated 1 +K factors in PCA with K dimensions in a MDS analysis when the data 
being analyzed were simulated to include one general factor plus K group factors, and showed 
that K group factors in PCA corresponded K dimensions in MDS. Again, notice that dimensions 
in MDS are major profiles in PAMS. The first principal factor in PCA can be interpreted as 
Spearman’s g factor or general factor in human intelligence tests or as trivial response bias 
(Hanson, Prediger, & Schussel, 1977), such as individuals’ particular response patterns, in 
interest/attitude assessment scales, and this general factor has usually high but equivalent 
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loadings across stimuli. When Davison (1985) reported the relationship between PCA and MDS, 
he did not develop the PAMS model, and could not include psychometric property of the general 
factor regarding MDS until in 1996. Now we propose that this general factor in the factor model 
corresponds to the level parameter in the PAMS model. 

In consistent with the constraints of the PAMS model, all parameters on these K latent 
variables are allowed to vary freely with the constraint that the parameters must sum to zero on 
each latent variable. This constraint has the solution uniquely identified. Further, the last K 
latent variables are constrained to be uncorrelated. The first latent variable accounts for 
individual differences in profile level. Along this latent variable, all observed variables are 
constrained to have equal weights since the first latent variable corresponds to the first principal 
factor in the factor model (see pp. 95-96, Davison, 1985). The first latent variable need not be 
uncorrelated to the other K latent variables, but it is left free to be correlated with them. In this 
paper, exploratory major profile patterns defined by the PAMS model are fitted to the data. 

Method 

The data used in this study involves the Basic Theme Scales from the Strong-Campbell 
Interest Inventory (Campbell & Hansen, 1985). The scales consist of the Realistic (REAL), 
Investigative (INVES), Artistic (ART), Social (SOCIAL), Enterprising (ENT), and Conventional 
(CONV) interest scales. Holland (1973) proposed a two-dimensional, hexagonal model for the 
six scales and this two-dimensional solution was interpreted as People vs. Things and Data vs. 
Ideas by Prediger (1982) and Tracey & Rounds (1993). The data for the study were collected by 
Rene Dawis and David J. Weiss from 1308 males who were clients of the Vocational 
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Assessment Clinic at the University of Minnesota. Table 1 shows the intercorrelations of the six 
interest scales. 



Insert Table 1 about here. 



For an exploratory PAMS, a nonmetric MDS was used to analyze these correlations and 
resulted in the two-dimensional solution. Since (see p.l05, Davison, 1993), 

where t and t’ refer to subscales, dissimilarity between two tests is inversely related with 
correlation. A confirmatory factor analysis, based on the same intercorrelations used in the 
exploratory PAMS approach, was performed through LISREL on the six subscales of the Strong- 
Campbell Interest Inventory. The hypothesized model is presented in Figure 1 where circles 
represent latent variables, and rectangles represent observed variables (or subscales). 

A three-factor model of “Interest,” Factor 1, General factor (which corresponds to the 
level parameter in the PAMS model and two group factors. Factor 2 and Factor 3, is 
hypothesized. Factor 2 and Factor 2 correspond the first and the second major profiles in PAMS, 
respectively. All six observed variables, which are REAL, ENVES, ART, SOCIAL, ENT, and 
CONV, serve as indicators of Factor 1, General factor. ENVES, ART, ENT, and CONV serve as 
indicators of Factor 2. REAL, ENVES, SOCIAL, and ENT serve as indicators of Factor 3. 
According to the assumptions of the PAMS model, last two factors are constrained to be 
uncorrelated with one another, but the first (general) factor is left free to be correlated with the 
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Results 

Model Estimation 

Unweighted least-squares estimation was employed to estimate all models. The 
hypothesized model (see Figure 1) was tested without inclusion of error covariances among 
observed variables (or subscales) but was not supported for the model, % (1 1, N = 1308) = 
240.57, P- value = .00, RMSEA = .13, AIC = 264.57, and GFI = .96. 

The model modifications were examined in an attempt to develop a better fitting, and 
possibly more parsimonious model. On the basis of the modification indices, the error 
covariances were added between ART and REAL, between SOCIAL and INVES, between ENT 
and INVES, between ENT and ART, and between CONV and ART. The error correlations for 
ART and REAL, for SOCIAL and INVES, for ENT and INVES, for ENT and ART, and for 
CONV and ART were -.25 (.03), .16 (.04), -.16 (.04), -.15 (.04), and -.26 (.04), respectively and 
all of them were significant at « = .001. The values in parentheses are standard errors of the 
estimates. The modified model that allowed the error correlations was supported and provided, 
(6, N = 1308) = 7.90, P-value = 0.25, RMSEA = 0.02, AIC = 41.90, CFl = 1.00, and GFI = 
1 . 00 . 

Parameter Estimation 

Table 2 shows the two-dimensional solution resulting from a nonmetric MDS analysis of 
the correlations among Interest Subscales. Dimension 1 was similar to Prediger’s (1982) Data 
vs. Ideas Dimension. According to theory, the Data end of the dimension should be marked by 
the Enterprising and Conventional scales that fall at the positive end of Dimension 1. The Ideas 
end should be marked by the Artistic and Investigative scales. Both fall at the negative end of 
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Dimension 1, although the Investigative scale does not fall as far toward the negative end as 
theory would lead one to expect. 



Insert Table 2 about here. 



According to theory, the People end of Dimension 2, the People vs. Things dimension, 
should be marked by the Social and Enterprising scales that fall at the negative end of Dimension 
2. The Things end should be marked by the Realistic and Investigative scales falling at the 
positive end of Dimension 2. 

By inspection of each dimension’s profile pattern, Davison et al. (2001) renamed the 
Data vs. Things dimension as Enterprising/Conventional (or E/C) Profile in the intent of 
interpreting a dimensional pattern, since Enterprising and Conventional subscales marked high 
points in the plot of the Data vs. Things dimension, and also relabeled the People vs. Things 
dimension as Realistic/Investigative (or R/I) Profile since these two subscales marked high 
points in the plot of the People vs. Things dimension. 

For SEM, three latent factors were assigned. Factor 1, Factor 2, and Factor 3, to the six 
Interest Subscales. Loadings of all six subscales on Factor 1 that accounts for the heights of 
individuals’ observed profiles in PAMS were constrained to be equal. LISREL assigned 0.58 to 
the six observed variables for Factor 1. According to the results of the dimensional solution in 
PAMS, INVES, ART, ENT, and COV were assigned to Factor 2 that corresponds to 
Enterprising/Conventional Profile. The second factor loadings on INVES, ART, ENT, and COV 
were -0.05 (.05), -0.51 (.05), 0.27(.05), and 0.29 (.05), respectively. The values in parentheses 
are standard errors of the estimates. Except the loading of INVES, the loadings of ART, ENT, 
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and COV were significant at a = .001. REAL, INVES, SOCIAL, and ENT were assigned to 
Factor 3 that corresponds to Realistic/Investigative Profile. The third factor loadings on REAL, 
INVES, SOCIAL, and ENT were 0.33 (.03), 0.34 (.04), -0.47 (.04), and -0.20 (.04), and all these 
loadings were significant at a = .001. 

According to the assumptions of the PAMS model. Factor 2 and Factor 3 were 
constrained to be uncorrelated, but Factor 1 (level parameter factor) was left free to be correlated 
with the other two factors. The correlations for Factor 1 and Factor 2 and for Factor 1 and Factor 
3 were 0.17 (.04) and 0.16 (.03), respectively, and both were statistically significant at a - .001. 

Figure 2 shows the path diagram between six observed variables (Interest Subscales) and 
three latent factors. Table 3 includes the weight of the three latent factors in SEM and scale- 
values of two major profiles (E/C & R/I Profiles). 



Insert Figure 2 and Table 3 about here. 



Although the scale-values in PAMS were different from the factor weights in SEM since 
different parameterization was used for each approach, the direction and magnitude of the scale- 
values were replicated in SEM. Figure 3 shows similar patterns between Factors 2 and 3 in SEM 
and Major Profiles 1 and 2 in PAMS. 



Insert Figure 3 about here. 
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Discussion 

Profile Analysis via Multidimensional Scaling (PAMS) is an exploratory technique 
designed to extract the major profile patterns from data of interest. It also estimates each 
individual’s observed score profile based on the major profiles identified in PAMS. Specifically, 
PAMS quantifies the direction and magnitude of the match between the major profile patterns 
and the observed score profiles of people by regressing actual test scores of individuals to the 
major profiles. 

The profile patterns identified by PAMS can be used to examine the relationship between 
the profile patterns and other criterion variables (Kim, Frisby, & Davison, 2001c) or used in 
regression (e.g., Kuang, 1998). Kim et al. (2001c) examined that the relationship between 
cognitive profile patterns and achievement test results in WJ-R data and found that there were 
significant relationships between the cognitive profile patterns and the achievement test results. 
Moreover, the level parameter was more strongly related with the achievement scores than the 
profile patterns. In this case the level parameter can be interpreted as Spearman’s g or general 
cognitive ability factor. 

However, the results from a PAMS approach will only serve as an exploratory tool. The 
profile patterns identified in PAMS is exploratory, but those profile patterns can be examined in 
confirmatory analyses illustrated by this paper. The hypothesized model, according to the 
assumptions of the PAMS model, which did not include the error covariances between subscales 
was not supported. Based on the modification indices recommended in the SEM analyses, the 
error covariances were included for: ENT and INVES (r = .13), ART and REAL (r = .06), 

CONV and ART (r = -.09), SOCIAL and INVES (r = .32), and ENT and ART (r = .01), and then 
the model was supported (P-value for Chi-square = .25). Inclusion of the error paths among the 
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subscales made it possible to fit the data to the model. Note that the values in parentheses are 
correlation coefficients between subscales. 

However, in general, the error terms should not correlated from one indicator to another. 
This is part of the definition of indicators of a construct. If the error terms for two or more 
indicators correlate, it means that the indicators measure something else in addition to the 
construct they are supposed to measure. If this is the case, the meaning of the construct and its 
dimensions may be different from what is intended. 

Because of that concern, we examined intercorrelations between the subscales added with 
error paths. As shown above, it is interesting to note that all other correlations were trivial, 
except the one between SOCIAL and INVES (r = .32), although the error correlations were all 
highly significant. From these results, one can suspect the measurement error between the 
PAMS approach and the SEM method when the results of PAMS analyses were attempted to be 
mapped onto the SEM scheme to replicate the results. When this measurement error was 
considered by including the error paths of the subscales in SEM, the hypothesized factor 
structure (the level parameter factor plus two dimensional factors) was replicated in the SEM 
analysis. 

While not introduced here, Kim (1999), Kim & Davison (2001a), Kim & Davison 
(2001b), and Kim, Craig, & Davison (2001c) have applied a bootstrapping technique to 
estimating standard errors of test parameters (i.e., MDS scale-values) in the PAMS model. In 
their simulation study, Kim & Davison (2001a) measured utility of the bootstrap procedure in 
estimating MDS scale-value standard errors, and reported that on average, irrespective of 
simulated conditions, the bootstrap error estimation included 80% of accuracy. Moreover, using 
the bootstrap procedure, Kim & Davison (2001b) proposed how to estimate standard errors of 
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correspondence indices (or person parameters) that quantify the match between individuals’ 
actual score profiles and major profiles in PAMS. 

Estimating standard error of any parameters of interest has at least two important aspects 
in PAMS analyses. First, it can help identify statistically significant scale-value estimates, and 
then allows users to include only significant values in interpreting major profiles identified in 
PAMS. Considering the utility of standard error estimates, future research should include 
(bootstrap) standard error estimates of scale-values/coordinates in major profiles extracted in 
PAMS, identify significant coordinates, and include only significant ones in interpretation of the 
major profiles. In addition to that, when the major profiles identified in PAMS are replicated in 
SEM, one can examine whether significant test results of the coordinates in PAMS match those 
of factor loadings in SEM since SEM analyses always provide asymptotic standard errors 
corresponding to estimates of parameters. 

This comparison allows us to check either reliability of bootstrap error estimate in PAMS 
or asymptotic error estimate in SEM when either of them is fixed to be a criterion. If there exits 
discrepancy between the bootstrap and the SEM error estimate, one may suspect that the 
asymptotic estimates would be underestimated. Regarding this issue, Kim (1999) and Weinberg, 
Carroll, and Cohen (1984) compared MDS scale-value’s asymptotic error (estimated the 
maximum likelihood method: ML) with bootstrap error estimates, and they consistently found 
that the ML estimates were significantly much more different from Monte Carlo results than the 
bootstrap estimates and the significant differences came mostly from underestimation. 
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Table 1 

Interest Inventory Intercorrelations among Males: Minnesota Vocational Assessment Clinic Data 
IN = 13081 



REAL INVES ART SOCIAL ENT CONV 



Realistic 


1.00 


0.52 


0.06 


0.18 


0.30 


0.37 


Investigative 


0.52 


1.00 


0.33 


0.32 


0.13 


0.37 


Artistic 


0.06 


0.33 


1.00 


0.26 


0.01 


-0.09 


Social 


0.18 


0.32 


0.26 


1.00 


0.38 


0.34 


Enterprising 


0.30 


0.13 


0.01 


0.38 


1.00 


0.36 


Conventional 


0.37 


0.37 


-0.09 


0.34 


0.46 


1.00 



Note . Data were collected by David J. Weiss and Rene Dawis, University of Minnesota, 
Vocational Assessment Clinic. 
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Table 2 

MDS Scale-values from Two-Dimensional Nonmetric Solution based on Intercorrelations of 
Interest Subscales 



Subscales 


Dimension 1 


Dimension 2 


Realistic 


0.11 


1.21 


Investigative 


-0.25 


0.96 


Artistic 


-2.14 


-0.05 


Social 


-0.07 


-1.14 


Enterprising 


1.09 


-.94 


Conventional 


1.26 


-0.03 
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Table 3 

Factor Loadings in SEM and Scale-values in PAMS 



Subscales 


Factor 1 (Level Parameter) 


Factor 2 (E/C Profile) 


Factor 3 (R/I Profile) 


Realistic 


0.58 (0.00) 


0.00 (0.11) 


1.01 (1.21) 


Investigative 


0.58 (0.00) 


-0.21 (-0.25) 


1.04 (0.96) 


Artistic 


0.58 (0.00) 


-2.15 (-2.14) 


0.00 (-0.05) 


Social 


0.58 (0.00) 


0.00 (-0.07) 


-1.44 (-1.14) 


Enterprising 


0.58 (0.00) 


1.14(1.09) 


-0.61 (-0.94) 


Conventional 


0.58 (0.00) 


1.22(1.26) 


0.00 (-0.03) 



Note . Factor 2 and Factor 3 loadings were multiplied by 4.21 and 3.07, respectively, to be 
consistent with magnitude of scale-values in PAMS analyses. 
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Figure Captions 

Figure 1. The Hypothesized Model in SEM by The Assumptions of the PAMS model 
Figure 2. The Modified Model Allowed With Error Covariances Between Interest Subscales 
Figure 3. Factor Profiles (Squares) in SEM Superimposed On Major Profiles (Circles) in PAMS 
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Figure 3. Factor Profiles (Squares) in SEM Superimposed On Major Profiles (Circles) in PAMS 



Factor 2 vs. E/C Profile (Dim 1) 




Factor 3 vs. R/l Profile (Dim 2) 




Note . Squares represent factor weights in SEM and circles represent scale-values in PAMS. 
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